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Abstract 



We propose a new approach to the analysis of Loopy Belief Propagation (LBP) by 
establishing a formula that connects the Hessian of the Bethe free energy with the 
edge zeta function. The formula has a number of theoretical implications on LBP. 
■ It is applied to give a sufficient condition that the Hessian of the Bethe free energy 

05 | is positive definite, which shows non-convexity for graphs with multiple cycles. 

O . The formula clarifies the relation between the local stability of a fixed point of 

LBP and local minima of the Bethe free energy. We also propose a new approach 
to the uniqueness of LBP fixed point, and show various conditions of uniqueness. 

> 

O ■ 1 Introduction 

CO 

. Pearl's belief propagation [1] provides an efficient method for exact computation in the inference 

CNI ' with probabilistic models associated to trees. As an extension to general graphs allowing cycles, 

Loopy Belief Propagation (LBP) algorithm [2] has been proposed, showing successful performance 
in various problems such as computer vision and error correcting codes. 



One of the interesting theoretical aspects of LBP is its connection with the Bethe free energy Q. It 
is known, for example, the fixed points of LBP correspond to the stationary points of the Bethe free 
energy. Nonetheless, many of the properties of LBP such as exactness, convergence and stability are 
' still unclear, and further theoretical understanding is needed. 



This paper theoretically analyzes LBP by establishing a formula asserting that the determinant of 
the Hessian of the Bethe free energy equals the reciprocal of the edge zeta function up to a positive 
factor. This formula derives a variety of results on the properties of LBP such as stability and 
uniqueness, since the zeta function has a direct link with the dynamics of LBP as we show. 

The first application of the formula is the condition for the positive definiteness of the Hessian of 
the Bethe free energy. The Bethe free energy is not necessarily convex, which causes unfavorable 
behaviors of LBP such as oscillation and multiple fixed points. Thus, clarifying the region where 
the Hessian is positive definite is an importance problem. Unlike the previous approaches which 
consider the global structure of the Bethe free energy such as 03 15], we focus the local structure. 
Namely, we provide a simple sufficient condition that determines the positive definite region: if all 
the correlation coefficients of the pseudomarginals are smaller than a value given by a characteristic 
of the graph, the Hessian is positive definite. Additionally, we show that the Hessian always has a 
negative eigenvalue around the boundary of the domain if the graph has at least two cycles. 

Second, we clarify a relation between the local stability of a LBP fixed point and the local structure 
of the Bethe free energy. Such a relation is not necessarily obvious, since LBP is not the gradient 
descent of the Bethe free energy. In this line of studies, Heskes [6| shows that a locally stable fixed 
point of LBP is a local minimum of the Bethe free energy. It is thus interesting to ask which local 
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minima of the Bethe free energy are stable or unstable fixed points of LBP. We answer this question 
by elucidating the conditions of the local stability of LBP and the positive definiteness of the Bethe 
free energy in terms of the eigenvalues of a matrix, which appears in the graph zeta function. 

Finally, we discuss the uniqueness of LBP fixed point by developing a differential topological result 
on the Bethe free energy. The result shows that the determinant of the Hessian at the fixed points, 
which appears in the formula of zeta function, must satisfy a strong constraint. As a consequence, 
in addition to the known result on the one-cycle case, we show that the LBP fixed point is unique 
for any unattractive connected graph with two cycles without restricting the strength of interactions. 

2 Loopy belief propagation algorithm and the Bethe free energy 

Throughout this paper, G = (V, E) is a connected undirected graph with V the vertices and E the 
undirected edges. The cardinality of V and E are denoted by N and M respectively. 

In this article we focus on binary variables, i.e., xi E {±1}- Suppose that the probability distribution 
over the set of variables x = (x,*),* e y is given by the following factorization form with respect to G: 

P( x ) = 7? II ^ij( x U x j) II ^i( x i)> w 
ijeE ieV 

where Z is a normalization constant and and ipi are positive functions given by tpij (xi ,Xj) = 
) and ipi(xi) — exp(hiXi) without loss of generality. 

In various applications, the computation of marginal distributions Pi(xt) :— J2 x \{x }P( X ) anc ^ 
Pij(xi,Xj) := J2x\{x x }P( X ) is required though the exact computation is intractable for large 
graphs. If the graph is a tree, they are efficiently computed by Pearl's belief propagation algorithm 
[1 J. Even if the graph has cycles, it is empirically known that the direct application of this algorithm, 
called Loopy Belief Propagation (LBP), often gives good approximation. 

LBP is a message passing algorithm. For each directed edge, a message vector fM^j (xj ) is assigned 
and initialized arbitrarily. The update rule of messages is given by 

Vi™( x j) oc ^2^ ji (x j ,x l )^ l (x i ) Yl Vk-n( x i), (2) 
Xi keNi\j 

where Ni is the neighborhood of i G V. The order of edges in the update is arbitrary. In this paper 
we consider parallel update, that is, all edges are updated simultaneously. If the messages converge 
to a fixed point {p.^j}, the approximations of pi(xt) and pij(xi,Xj) are calculated by the beliefs, 

bi(xi) (x i>i{xi) Y\ (xi,Xj) oc il)ij(xi,Xj)ipi(xi)tl;j(xj) Yl VkUi(xi) Yl VkUj(xj), 

(3) 

with normalization J2x ^i( x i) = 1 an d J2x x- ^ij{ x ii x j) — 1- From (0 and OJ, the constraints 
bij(xi, Xj) > and ^ij( x ii x i) = ^>i{ x i) are automatically satisfied. 

We introduce the Bethe free energy as a tractable approximation of the Gibbs free energy. The 
exact distribution ([TJ is characterized by a variational problem p(x) — argmin^ Fcmsip), where 
the minimum is taken over all probability distributions on (xi)i^v an d Fabbsip) is the Gibbs free 
energy defined by Fcms (p) = KL(p\ \p) — log Z. Here KL{p\ \p) — J p log(p/p) is the Kullback- 
Leibler divergence from p to p. Note that Fcms {p) is a convex function of p. 

In the Bethe approximation, we confine the above minimization to the distribution of the form 
b(x) oc IlijeB bij(xi, Xj) Yliev biixi) 1 ^^ , where di :— \Ni\ is the degree and the constraints 
bij(Xi,Xj) > 0, J2x i: x = 1 and ^2 X . b ij (x l , xj) = bi(xi) are satisfied. A set 

{bi(xi), bij(xi, Xj)} satisfying these constraints is called pseudomarginals. For computational 
tractability, we modify the Gibbs free energy to the objective function called Bethe free energy: 

F(b) := - y^ j b i j{x i ,Xj) log ipjj (x j , Xj ) - bj (Xj ) log ipj (Xj ) 

ij£E XiXj i£V Xi 

+ ^2 bij(xj, Xj)logbij(xi,Xj) + y^(l — dj)'Y]bi(xi)logbi(xi). (4) 

ijGExiXj i£V Xi 
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The domain of the objective function F is the set of pseudomarginals. The function F does not 
necessarily have a unique minimum. The outcome of this modified variational problem is the same 
as that of LBP [3 1. To put it more precisely, There is a one-to-one correspondence between the set 
of stationary points of the Bethe free energy and the set of fixed points of LBP. 

It is more convenient if we work with minimal parameters, mean m, = Ej )i [xi] and correlation 
Xij — E& w [ariXj]. Then we have an effective parametrization of pseudomarginals: 

bij[x^Xj) = -(1 + rriiXi + rrijXj + Xij%i%j), — ^(1 + m,-). (5) 

The Bethe free energy (O is rewritten as 

F({m,Xij}) = - / ] JijXi] - himi 

ijeE iev 

+ LLH p — - — -J + z^-^LH — 2 — r () 

where rj(x) :— x log x. The domain of F is written as 

L(G) := \^{m l ,Xij} £ R JV+M |1 + mja;i +m,jXj +Xij%iXj > OforalHj 6 E and Xi,Xj = ±lj- 

The Hessian of F, which consists of the second derivatives with respect to {rrii, Xij}> is a square 
matrix of size N + M and denoted by V 2 F. This is considered to be a matrix-valued function on 
L(G). Note that, from (0, V 2 F does not depend on and /ij. 



3 Zeta function and Hessian of Bethe free energy 
3.1 Zeta function and Ihara's formula 

For each undirected edge of G, we make a pair of oppositely directed edges, which form a set of 
directed edges E. Thus \E\ = 2M, For each directed edge e e E, o(e) £ is the origin of e and 
t(e) E is the terminus of e. For e £ E, the inverse edge is denoted by e, and the corresponding 
undirected edge by [e] = [e] E £7. 

A closed geodesic in G is a sequence (ei,...,efe) of directed edges such that t(ei) = 
o(ei + i) and ej 7^ e^+i for i E Z/fcZ. Two closed geodesies are said to be equivalent if one is 
obtained by cyclic permutation of the other. An equivalent class of closed geodesies is called a 
prime cycle if it is not a repeated concatenation of a shorter closed geodesic. Let P be the set of 
prime cycles of G. For given weights u = (u e ) ee g, the edge zeta function QO is defined by 

Cg?(«) := -S(p)r\ 9(P) ■= u ei ■ ■ ■ u ek forp = (e u ...,e k ), 

PGP 

where u e E C is assumed to be sufficiently small for convergence. This is an analogue of the 
Riemann zeta function which is represented by the product over all the prime numbers. 

Example 1. If G is a tree, which has no prime cycles, (g( u ) = !■ For 1-cycle graph Cm of 
length JV", the prime cycles are (ei,e2, . . . , ejv) and (ejy, ejv-i, ■ • ■ , ei), and thus (c N ( u ) = (1 — 

Ilti u ei )~ 1 (1 — Ll^i u si )~ 1 ■ Except for these two types of graphs, the number of prime cycles is 
infinite. 

It is known that the edge zeta function has the following simple determinant formula, which gives 
analytical continuation to the whole C 2M . Let C(E) be the set of functions on the directed edges. 
We define a matrix on C(E), which is determined by the graph G, by 

fl ife^e>and (e)=*(e'), 
e,e \0 otherwise. 

Theorem 1 ([8 1, Theorem 3). 

Ca(u) =det{I-UM)- 1 , (8) 
where IA is a diagonal matrix defined by U e . e i := u e 5 eje >. 
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We need to show another determinant formula of the edge zeta function, which is used in the proof 
of Theorem[3] We leave the proof of Theorem[2]to the supplementary material. 
Theorem 2 (Multivariable version of Ihara's formula). Let C(V) be the set of functions on V. We 
define two linear operators on C(V) by 

(£/)(*') := ( E T^M/«, Uf)(i) ■= E r^^/Ke)), where f G C(V). 

t(e)=« *(«)=< 

(9) 

TTzen we /zave 

(Cg(«) _1 = ) det(J - WM) = det(/ + Z> - A) J| (1 - w e w g ). (10) 

[e]e-E 

If we set u e = u for all e G -E , the edge zeta function is called the Ihara zeta function J9) and 
denoted by Cg(u). In this single variable case, Theorem|2]is reduced to Ihara's formula ifTOl : 

2 

Cciu)- 1 = det(J - uM) = (1 - u 2 ) M det(J + -^V - -^.4), (11) 

1 — u z 1 — u z 

where T> is the degree matrix and .4 is the adjacency matrix defined by 

(2>/)(i) := */(*), W)(i):= E /(°( e ))' / eC< (n 

ee.E,t(e)=i 

3.2 Main formula 

Theorem 3 (Main Formula). The following equality holds at any point of L(G): 
(C G («) _1 =)dct(7-WA^) ^dettV 2 ^) [J [| ^(a*, a,-) J] J] fcjfc) 1 -* 2 

where 6y anc/ fe^ are given fey Q a«J 

Xi? Tfl^TTbj 



2JV+4M 



"•l—tj ■ — o 

l — TUj 

Proof. (The detail of the computation is given in the supplementary material.) 

From ©, it is easy to see that the (E,E)-block of the Hessian is a diagonal matrix given by 

d 2 F If 1 1 1 1 

OijM — I — ; ; h ; ; 1~ 



(12) 
(13) 



dXijdXki ' 4:\l + mi + m,j+Xij l — nn+mj—Xij l + rm — rnj—Xij l — mi — rrij+Xij' 
Using this diagonal block, we erase (V,E)-block and (E,V)-block of the Hessian. In other words, 
we choose a square matrix X such that dct X = 1 and 

Y 







X T (V 2 F)X 

After the computation given in the supplementary material, we see that 



(Xafc~ m i m fc) 'f ■ ■ 

f V \ _ J i-m; ' l^keNi (l-mJ)(l-mJ-m^+2m i m fc Xifc-X?fc) * ~~ ] ' (\ A\ 

( h ' J ~ S -A j , i x il" m<mj ' r otherwise. 



From itj^i = x ' 3 1 _™'™ 3 . it is easy to check that 7jy + 2? — .4 = YW, where A and 2? is defined in 
(0 and W is a diagonal matrix defined by Wij := <5,;.j(l — mf). Therefore, 

det(I - UM) = det(Y) (1 - m 2 ) JJ (1 - u e u s ) = R.H.S. of fl2j 

For the left equality, Theorem|2]is used. □ 

Theorem[3]shows that the determinant of the Hessian of the Bethe free energy is essentially equal to 
det(7— UM), the reciprocal of the edge zeta function. Since the matrix UM has a direct connection 
with LBP as seen in section 5, the above formula derives many consequences shown in the rest of 
the paper. 
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4 Application to positive definiteness conditions 



The convexity of the Bethe free energy is an important issue, as it guarantees uniqueness of the fixed 
point. Pakzad et al ifTTl and Heskes [5 1 derive sufficient conditions of convexity and show that the 
Bethe free energy is convex for trees and graphs with one cycle. In this section, instead of such 
global structure, we shall focus the local structure of the Bethe free energy as an application of the 
main formula. 

For given square matrix X, Spec(X) C C denotes the set of eigenvalues (spectra), and p(X) the 
spectral radius of a matrix X, i.e., the maximum of the modulus of the eigenvalues. 
Theorem 4. Let A4 be the matrix given by (0- For given {rrii, Xij} S L(G), U is defined by M3h 
Then, Spec(W.M) C C\R>i ==>• V 2 F is a positive definite matrix at {rrii,Xij}- 

Proof. We define mi(t) :— rrii and Xij (*) : — tXij + (1 — t)rriimj. Then {mi(t), Xij(i)} G L(G) 
and {nii(l), Xij(l)} — { m i, Xij}- For t <E [0, 1], we define U(t) and \7 2 F(t) in the same way by 
{mi(i), x»j (*)}• We see that W(i) =tU. Since Spec(UM) C C\M> X , we have det(I-tUM) ^0 
v t S [0, 1], From Theorem|3] det(V 2 F(i)) / holds on this interval. Using O and Xij(0) = 
mi(0)m_y (0), we can check that V 2 F(0) is positive definite. Since the eigenvalues of V 2 F(t) are 
real and continuous with respect t, the eigenvalues of \7 2 F(1) must be positive reals. □ 

We define the symmetrization of and Uj^i by 

Xij-rrHmj _ CavbylxuXj] 
Pl " J Pl " 1 ' {(1 - m? )(1 - m^)}Va {Var bi [ Xi ] Var 6 . " ^ D} 

Thus, Ui^jUj^i = fii^jflj^i. Since /?i->j = /3j->-i, we sometimes abbreviate ft-^j as From 
the final expression, we see that \f$ij\ < 1. Define diagonal matrices Z and B by (-Z) e ,e' := 
5 e ,e'(l — m t( e )) 1 ^ 2 an ^ (^)e,e' : = <5e,e'/3 e respectively. Then we have -ZW.MZ -1 = £>.M, because 

(^2-')^ - (1 - m\ e) yl 2 u e {M)^(\ - m 2 o{e) r 1/2 = Pe(M) e , e <. 
Therefore Spec(^X) = Spcc(BM). 

The following corollary gives a more explicit condition of the region where the Hessian is positive 
definite in terms of the correlation coefficients of the pseudomarginals. 

Corollary 1. Let a be the Perron Frobenius eigenvalue of M and define L a -i(G) := {{mi, Xij} £ 
L(G)||/3 e | < a -1 for all e € E}. Then, the Hessian V 2 F is positive definite on L a -i(G). 

Proof Since |/3 e | < a" 1 , we have p{BM) < p(a _1 JW) = 1 (Q2 Theorem 8.1.18). Therefore 
Spec(i3A^)nM>i = 0. □ 

As is seen from (fTTT i. a -1 is the distance from the origin to the nearest pole of Ihara's zeta £g(u). 
From exampleQ] we see that Cg(u) = 1 f° r a ti" ee G and (c N (u) = (1 — u N )~ 2 for a 1-cycle graph 
Cat. Therefore a -1 is oo and 1 respectively. In these cases, L a -\(G) = L(G) and F is a strictly 
convex function on L(G), because |/3 e | < 1 always holds. This reproduces the results shown in IfTTl . 
In general, using Theorem 8.1.22 of lfT2l . we have min^y di — 1 < a < max^y di — 1. 

Theorem|3]is also useful to show non-convexity. 

Corollary 2. Let {rrii(t) :— 0, Xijif) := *} £ L{G)jbr t < 1. 77ie« we Ziave 

limdet(V 2 F(t))(l - f) M + w -i = _2- M " Ar + 1 (M - W)«(G), (16) 

where k(G) is the number of spanning trees in G. In particular, F is never convex on L{G) for any 
connected graph with at least two linearly independent cycles, i.e. M — N > 1. 

Proof. The equation ([TBT l is obtained by Hashimoto's theorem [13], which gives the u — > 1 limit 
of the Ihara zeta function. (See supplementary material for the detail.) If M — N > 1, the right 
hand side of ( [T6| > is negative. As approaches to {mj = 0, Xij = 1} G L{G), the determinant of the 
Hessian diverges to — oo. Therefore the Hessian is not positive definite near the point. □ 

Summarizing the results in this section, we conclude that F is convex on L(G) if and only if G is a 
tree or a graph with one cycle. To the best of our knowledge, this is the first proof of this fact. 
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5 Application to stability analysis 



In this section we discuss the local stability of LBP and the local structure of the Bethe free energy 
around a LBP fixed point. Heskes [6| shows that a locally stable fixed point of sufficiently damped 
LBP is a local minima of the Bethe free energy. The converse is not necessarily true in general, and 
we will elucidate the gap between these two properties. 

First, we regard the LBP update as a dynamical system. Since the model is binary, each message 
Hi-^.j(xj) is parametrized by one parameter, say rji^j. The state of LBP algorithm is expressed 
by V = (Ve) ee g e C(E), and the update rule (O is identified with a transform T on C(E), 
ri ncw = T(rf). Then, the set of fixed points of LBP is {rj 00 E C(E)\T{rj°°) = rj°°}. 

A fixed point 77°° is called locally stable if LBP starting with a point sufficiently close to tj°° con- 
verges to 77°°. The local stability is determined by the linearizion T' around the fixed point. As is 
discussed in [14), r?°° is locally stable if and only if Spe^T'^ 00 )) C {A £ C||A| < 1}. 

To suppress oscillatory behaviors of LBP, damping of update T € :— (1 — e)T + el is sometimes 
useful, where < e < 1 is a damping strength and / is the identity. A fixed point is locally stable 
with some damping if and only if Spec(T' (r}°°)) C {A e C|RcA < 1}. 

There are many representations of the linearization (derivative) of LBP update (see |[T4l Q3)), we 
choose a good coordinate following Furtlehner et al lfl6l . In section 4 of [ 16], they transform mes- 
sages as — > fii^j / fj,°^.j and functions as t/V, — > bij/(bibj) and tpi — > hi, where is 
the message of the fixed point. This changes only the representations of messages and functions, 
and does not affect LBP essentially. This transformation causes T'(r)°°) — > PT' (r] 00 )P~ 1 with an 
invertible matrix P. Using this transformation, we see that the following fact holds. (See supple- 
mentary material for the detail.) 

Theorem 5 ([ 16 1, Proposition 4.5). Let Ui^j be given by Q, @ and £73]) at a LBP fixed point rj°°. 
The derivative T'(r7°°) is similar to 1AM., i.e. UM = PT' (r]° )P~ 1 with an invertible matrix P. 

Since dct(7 — T (t]°°)) = dct(7 — UM), the formula in Theorem[3]implies a direct link between 
the linearization T (r)°°) and the local structure of the Bethe free energy. From Theorem|4j we have 
that a fixed point of LBP is a local minimum of the Bethe free energy if Spec(T'(j7°°)) C C \ R>i. 

It is now clear that the condition for positive definiteness, local stability of damped LBP and local 
stability of undamped LBP are given in terms of the set of eigenvalues, C \ R>i, {AG C|ReA < 1} 
and {A £ C||A| < 1} respectively. A locally stable fixed point of sufficiently damped LBP is a 
local minimum of the Bethe free energy, because {A e C|ReA < 1} is included in C \ M>i. This 
reproduces Heskes's result |6|. Moreover, we see the gap between the locally stable fixed points 
with some damping and the local minima of the Bethe free energy: if Spec(T'(j7°°)) is included in 
C \ M>i but not in {A € C|RcA < 1}, the fixed point is a local minimum of the Bethe free energy 
thougfit is not a locally stable fixed point of LBP with any damping. 

It is interesting to ask under which condition a local minimum of the Bethe free energy is a stable 
fixed point of (damped) LBP. While we do not know a complete answer, for an attractive model, 
which is defined by > 0, the following theorem implies that if a stable fixed point becomes 
unstable by changing Jy and hi, the corresponding local minimum also disappears. 

Theorem 6. Let us consider continuously parametrized attractive models {ipij(t),il>i(t)}, e.g. t 
is a temperature: ^>ij{t) = exp(i _1 JijXiXj) and ipi{t) — exp(t~ 1 hiXi). For given t, run LBP 
algorithm and find a (stable) fixed point. If we continuously change t and see the LBP fixed point 
becomes unstable across t — to, then the corresponding local minimum of the Bethe free energy 
becomes a saddle point across t = <o- 

Proof. From (0, we see b ) for some 6>; and 8j. From 

Jij > 0, we have Cov^. [xi, Xj] = \ij ~ miirij > 0, and thus u^j > 0. When the LBP fixed point 
becomes unstable, the Perron Frobenius eigenvalue of UM goes over 1, which means dct(7 — UM) 
crosses 0. From Theorem[3]we see that det(V 2 F) becomes positive to negative at t = to. □ 

Theorem|6]extends Theorem 2 of 1141 . which discusses only the case of vanishing local fields hi = 
and the trivial fixed point (i.e. rrij = 0). 
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6 Application to uniqueness of LBP fixed point 



The uniqueness of LBP fixed point is a concern of many studies, because the property guarantees that 
LBP finds the global minimum of the Bethe free energy if it converges. The major approaches to the 
uniqueness is to consider equivalent minimax problem [5], contraction property of LBP dynamics 
ifTTl IT8l . and to use the theory of Gibbs measure fl9l . We will propose a different, differential 
topological approach to this problem. 

In our approach, in combination with Theorem[3] the following theorem is the basic apparatus. 
Theorem 7. // det V 2 F{q) ± for all q G (VF)" 1 ^) then 

^ sgn (det V 2 -F(<7)) = 1, where sgn(x) := 

q:VF(q)=0 

We call each summand, which is +1 or — 1, the index of F at q. 

Note that the set (Vi* 1 ) -1 (0), which is the stationary points of the Bethe free energy, coincides with 
the fixed points of LBP. The above theorem asserts that the sum of indexes of all the fixed points 
must be one. As a consequence, the number of the fixed points of LBP is always odd. Note also that 
the index is a local quantity, while the assertion expresses the global structure of the function F. 

For the proof of Theorem [7] we prepare two lemmas. The proof of Lemma Q] is shown in the 
supplementary material. Lemma [2] is a standard result in differential topology, and we refer ||20l 
Theorem 13.1.2 and comments in p. 104 for the proof. 

Lemma 1. If a sequence {q n } C L(G) converges to a point q* G dL(G), then \\\7F(q n )\\ —> oo, 
where dL(G) is the boundary of L(G) C R N+M . 

Lemma 2. Let M\ and be compact, connected and orientable manifolds with boundaries. 
Assume that the dimensions of M% and M% are the same. Let f : M\ — > Mi be a smooth map 
satisfying f(d M\) C dMi. For a regular value of p G Mi, i.e. det(V/(g)) ^ for all q G f (p), 
we define the degree of the map f by degf := Ylq^f- 1 (p) sgn(det V/(q)). Then degf does not 
depend on the choice of a regular value p G M%. 

Sketch of proof. Define a map $ : L(G) -4 R N+M by $ := \7F + Note that $ does not 
depend on h and J as seen from (|6). Then it is enough to prove 

sgn(det V$(g)) = ^ sgn(det V$(g)), (17) 

9 e*- 1 ((^)) ge*-!(o) 

because $ -1 (0) has a unique element {mi = 0,Xij = 0}, at which V 2 F is positive definite, and 
the right hand side of (|45T > is equal to one. Define a sequence of manifolds {C„} by C n := {q G 
-^(C) I J2ij£E T^xi x~ 1°S &y — n )' which increasingly converges to L(G). Take K > and e > 
to satisfy K — e > ||(j)||- From LemmaQ] for sufficiently large no, we have $ _1 (0), (j) C 
C n „ and $(9C„ ) PI Bq(K) — <fi, where Bq(K) is the closed ball of radius K at the origin. Let 
IT e : M. N+ — > Bq (K) be a smooth map that is the identity on Bo (K — e), monotonically increasing 
on ||x||, and II e (x) = -^x for ||.t|| > K. We obtain a map $ := n c o $ : C„ ->■ B (K) such that 
<l(<9C no ) G 8Bq(K). Applying Lemma yields ©. □ 

If we can guarantee that the index of every fixed point is +1 in advance of running LBP, we conclude 
that fixed point of LBP is unique. We have the following a priori information for (3. 

Lemma 3. Let j3ij be given by M5j at any fixed point of LBP. Then \(3ij\ < tanh(|J^|) and 
Sgn(/%) = sgn(Jy ) hold. 

Proof. From (0, we see that bij(xi,Xj) oc exp(JijXiXj + 6iXi + 9jXj) for some 9i and 
8j. With ( fTBT l and straightforward computation, we obtain /3jj = sinh(2 Jy)(cosh(2^i) + 
cosh(2J. u ))- 1 / 2 (cosh(26' j ) + cosh(2J„ ))- :L / 2 . The bound is attained when t =0and^ =0. □ 

From Theorem|7]and Lemma[3] we can immediately obtain the uniqueness condition in 1181 . though 
the stronger contractive property is proved under the same condition in IfTSl . 



f 1 ifx > 0, 
1-1 ifx < 0. 
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Figure 1 : Graph of Example|2] Figure 2: Graph G. Figure 3: Two other types. 

Corollary 3 ( 11181 ). If p{JM) < 1, then the fixed point of LBP is unique, where J is a diagonal 
matrix defined by — tanh(| J e \)S e ,e>- 

Proof Since |/%| < tanh(|J y |), we have p(BM) < p{JM) < 1. (Q3 Theorem 8.1.18.) Then 
det(J - BM) = det(7 - UM) > implies that the index of any LBP fixed point must be +1. □ 

In the proof of the above corollary, we only used the bound of modulus. In the following case of 
Corollary |U we can utilize the information of signs. To state the corollary, we need a terminology. 
The interactions { J,j, hi} and { J/-, h^} are said to be equivalent if there exists (sj) 6 {±1} V such 
that Jtj = JijSiSj and h! i — /i$Sj. Since an equivalent model is obtained by gauge transformation 
Xi — > XiSi, the uniqueness property of LBP for equivalent models is unchanged. 

Corollary 4. If the number of linearly independent cycle ofG is two (i.e. M — N + 1 = 2), and the 
interaction is not equivalent to attractive model, then the LBP fixed point is unique. 

The proof is shown in the supplementary material. We give an example to illustrate the outline. 

Example 2. Let V := {1, 2, 3, 4} and E := {12, 13, 14, 23, 34}. The interactions are given by 
arbitrary {hi} and {— J12, J13, Ju, J23, J34} with > 0. See figure[TJ It is enough to check that 
det(7 - BM) > for arbitrary < /?i 3 , ^23,^14, /3 34 < 1 and -1 < /3i 2 < 0. Since the prime 
cycles of G bijectively correspond to those of G (in figure|2|i, we have det(J— BM) = det(7— BM), 
where /3 e ^=_/3 12j 23 , 4 2 = A3, and (3 f3 = /? 34 . We see that det(J - BM) = (1 - /3 ei /3 e2 - 
/? ei /3e 3 -/? e2 /3e 3 -2/3 ei /3 e2 /3 ea )(l-/3 ei /3 e2 -$ ei $e 3 ~$e 2 $e 3 + Wei Pe 2 K 3 ) > 0. In other cases, 
we can reduce to the graph G or the graphs in figure [3] similarly (see the supplementary material). 

For attractive models, the fixed point of the LBP is not necessarily unique. 

For graphs with multiple cycles, all the existing results on uniqueness make assumptions that up- 
perbound | | essentially. In contrast, Corollary [4] applies to arbitrary strength of interactions if the 
graph has two cycles and the interactions are not attractive. It is noteworthy that, from Corollary [2] 
the Bethe free energy is non-convex in the situation of CorollaryH] while the fixed point is unique. 

7 Concluding remarks 

For binary pairwise models, we show the connection between the edge zeta function and the Bethe 
free energy in Theorem[3] in the proof of which the multi-variable version of Ihara's formula (Theo- 
rem[5]i is essential. After the initial submission of this paper, we found that Theorem[3]is extended to 
a more general class of models including multinomial models and Gaussian models represented by 
arbitrary factor graphs. We will discuss the extended formula and its applications in a future paper. 

Some recent researches on LBP have suggested the importance of zeta function. In the context of the 
LDPC code, which is an important application of LBP, Koetter et al 12"T1 l22l show the connection 
between pseudo-codewords and the edge zeta function. On the LBP for the Gaussian graphical 
model, Johnson et al [23 1 give zeta-like product formula of the partition function. While these are 
not directly related to our work, pursuing covered connections is an interesting future research topic. 
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Supplementary Material for 

"Graph Zeta Function in the Bethe Free Energy and 
Loopy Belief Propagation" 

Yusuke Watanabe and Kenji Fukumizu, NIPS 2009 
A Proof of Theorem H 

Proof. First, we define three linear operators O : C(V) -> C(E), T* : C(E) -)• C(V), and 
L : C(E) ->■ C(E) as follows: 

(0/)(e):=/(o(e)), (T*g)(i) := E »( e )' fo)(e) := s(e) where / € C(V) and g G C(E). 

We see that = 0T* - t, because 

((OT* - i)g) (e) = £ s(e') - 9® = (Mg)(e) for g G C(£). 

e'G-B,t(e')=o(e) 

Then we have 

det(I-UM) =det (/ -UOT*{I + det(J + Wt) 

= det (i -T*{I + Ul)- 1 U(D} det(I + Ui). 

In the second equality, we used det(/„ — AS) = det(/ m — £M) for n X m and to x n matrices 
A and B fll24l . Lemma 8.2.4). The linear operator u is a block diagonal matrix with standard basis. 
The (e, e) block of I + Ui is 

1 u e 

Ug 1 

Therefore, we have det(I + Ui) = Y\^ eE {^ — u e u s ). 

Finally, we check that T*(I + Uij^UO = A - t>. The matrix (I + Ui)- 1 is a block diagonal 
matrix with (e, e) block 

1 

1 - u e u s 

For / G C(V), we have 

(r*(i + Ui)- 1 uof)(i)= E ({i +ut)- 1 uofj{e) 

eeE,t(e)=i 

= E T-T7Tr(( W °/)( e ) - MUOf)(e)) 

e£E,t(e)=i 

= E i ~ f "e/(o(e)) - u e u s f(o(e))) 

^— ' 1 — u e u s V / 

e£-E,t(e)=i 
= (^/)(i) -(©/)(*). 

□ 



1 

-Up 



1 



(18) 
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B Proof of Theorem|3] 



B.l Explicit formula of derivatives of the Bethe free energy 



In the proof of TheoremO the graph G = (V, E) is assumed to be a simple graph, i.e., there is no 
multiple edges and loop-edge 

For the proof, we need explicit expressions of the second derivatives of the Bethe free energy. We 
list them below. 



The first derivatives of the Bethe Free Energy are 

dF 

■ = -/); -!■■ II - <l. , ^ 



dm 

dF 

dXi. 



- = -hi + (1 - di)- ^2 Xilagbi(xi) + x t \ogb ik (xi,Xk), (19) 



x t =±l 



keNi x i: x k =±i 



J-ij ~t~ . ^ ^ XiXj log bij Xj)- 



Uj ' Xi,X j= ±l 

The second derivatives of the Bethe Free Energy are 

d 2 F 



dniidmj 



d 2 F 

dm k dxij 
d 2 F 



IV 

4 Z-^Xi,Xj 1 



1 

—m'j 


+ 4 *l2kENi ^2xi 






1+m 






Xj 


1+m 


Xi+rrij Xj+Xij x i x j 




Xi 


1+m 


Xi-\-rrijXj-\-Xij x i x j 




1 



dXijdxki [0 
We use notations 



: 1+rriiXi+mkXk+XikXiXk 

if i and j are adjacent ( 
otherwise, 

if k = i, 
if k = j, 
otherwise, 

if ij = kl, 
otherwise. 



if i = 



4 ^ 1 



tin •' " 



4 ^ 1 



1 + rriiXi - 


- rrijXj - 






Xj 




1 + niiXi - 


- rrijXj - 










1 + rriiXi - 


- TUjXj - 





(20) 

(21) 

(22) 
(23) 

(24) 
(25) 
(26) 



Note that r.y = and tij = tji, but Sjj ^ Sji in general. 
B.2 Detailed proof of Theorem|3] 

Proof. First, note that the Hessian of the Bethe free energy is a square matrix of size N + M; 

( a 2 F \ ( o 2 f y 



V 2 F({m t , Xl3 }) 



( d 2 F \ ( d 2 F \ 

\dxuvdmj J \dXuvdXst J 



^dxuvdmj j \dXuvdXst _ 

Recall that N is the number of vertices and M is the number of undirected edges. 
Stepl: Computation of Y 

From d23l , the (E,E)-block of the Hessian is a diagonal matrix given by 

d 2 F 

dXijdxki 

Using this diagonal block, we erase (V,E)-block and (E, V)-block of the Hessian. Thus, we obtain a 
square matrix X such that dct X = 1 and 

Y 

\OXijOXkl J _ 



X T (V 2 F)X 



11 



Applying an identity 



(\ 





ry \ (Wi 


tij 





1 


— ' 


Wj 


V> 





1 J 


Sji 



0\ 

1 
^ii 1 , 



for each edge, we have 



, — fiifii if i anc j j are adjacent , 
otherwise. 







The elements of Y are represented in terms of {m i: Xij} as follows: 

1 - 2 
— + 



(Y)i 



1 - m? 
1 



£ ('* - — - r^) 

^ r ife 1 - 

(Xifc - rn l m k ) 2 



I — mi 



y 

^ (1 - mf)(l - mf - m| + 2m i m k x l k - xf k ) 



and, 



fee at., 



Sij S j 



-(Xij - miirij) 



(1 - m? - m| + 2m l m j Xij - X%) 



for adjacent i and j. 



Step2: Computation of In + T> — A 

From the definition (fT~3T > of Uj^i, we see that 



{Xij ~ ■m, i m j ) 2 



\ in- ■ 2iii,iu - 1 

(l-m|)(x» 3 -m. t m 3 ) 
1 - Ui^jUj^i (1 - mf - m3 + 2m i m J Xij ~ X%) 



1 - Ui^jUj^i (1 - mf - m 2 + iTYLiTYLjXij - xfj) 



Therefore, the diagonal element is 

(I N + V - A)i,i = (In + =H^r 



(Xik ~ m.mk) 2 



keNi 



(l- m 2 -mt+2m l m k x l k-xtk) 



and for adjacent i and j, 

(/ w + v - 



<A) itj = — 



-(1 - m 2 )(xi 3 - mirrij) 
(1 - m 2 -m 2 + 2miirijXij - xl) ' 



Combining the results of step 1 and 2, we have 



In + V - A = Y 



\-m\ 



1 - m 2 



l-m N 
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Step3: Final step 
We see that 



(ciu)- 1 =det(I-UM) 

= det(7jv +V- A) Yl (1 - u e u e) 



e]£E 

UpUe 



^det^na^^) n 1 = u ^ ju ^ 

ieV ij£E lJ 

= det(V 2 F)n(l-m^ J] (1 '^^ )(1 .~ mf)(1 " m|) . 

From d27l i to (f28T >, we used the edge zeta version of Ihara's formula (Theorem|2]i. 
Furthermore, with a straightforward computation we see that 



(1 - Ui-yjUj^.i)(l - m?)(l - ra 2 ) 



4 4 J| bijfauXj), 



(l-m?) 1 -* = 2 a - 3 * J] W^*. 

an=±i 

where bij(xi,Xj) = |(1 + + m^- + Xij x i x j) and M^i) = |(1 + m 0- 
Therefore, 

=2 & e vP-2*) 4 ^det(v a F)n n ^w 1 -* n n m**.** 
: 2^+ 4M det(v 2 F)n n & i(^) i_d< n n m^o. 

ijeExi,Xj=±l 



C Proof of Corollary |2] 

Here, we prove the limit formula in Corollary[2] 

Proof. We can easily check that (t) = t, 

II II M^i) = 4" 4M (1 - t) 2M (l + tf M , and 

n n &i(^) i - d< =2- 2iv+4M 

i£VXi = ±l 

on this interval. Therefore 



Yimdet(V z F{t)){l-t) M+N -' = lini Co («(*))" (1 - *) 



(4- 4M (l-i) 2M (l+<) 
lirnCG^-^l-t)-^-^ 

t— ^1 

— (M — N)k(G)2~ 



2M2~2N+4M22N+4M 
-M+N-lr)-2M 
M-N+l 



On the final equality, we used Hashimoto's formula: 



lim Cg(u) _1 (1 - u)- M+JV - x = -2 M " JV+1 (M - JV)k(G). 

u— >1 



We refer to 1 13 25 26] for this formula. 
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D Transformation of messages and proof of Theorem 5 



D.l Transformation of messages 

First, we make an easy observation on the LBP update. 

Proposition 1. Let {iTi^j} be any set of messages. We define a transformation from messages 
{/4-k,} to messages by 



fJ-UM) n (r y ( 3 °) 



We also define transformation from functions {ipij, ipi\ to functions {ipij, ipi} by 

It \ Yij\ x ii x j) /tin 

ipijfaxj) <x f— — —r^> (3 V 

Ki^fj (Xj )TTj^i{Xi) 

4>i(xi) oc ipi(xi) Y[ K k ^i{xi). (32) 

keNi 



Then the update 
is equivalent to 



Xi k£Ni\j 

P^)(xj) oc ^^(xj^i^iixi) Y[ M/UiOi)- (34) 

Xi k£Ni\j 



Proof. The equivalence of ( 133) and (134) is easily checked by ( 130) , OTb , and ( 132) ■ □ 
Symbolically, Proposition[T]implies that 

noTor^f, (35) 

where II is the transformation of the messages by 7ij_).j, T is the LBP update with {ipij, ipi}, and 
T is the LBP update with {ipij,ipi} . Differentiation of this relation gives the transformation of the 
linearization matrix. 



If we choose {7Tj_ ^} as ir^jixj) — f J, i°>i then ( 130) , d 3 1 1 > and ( 132) becomes 

„oo (36) 



^fc ,»,-)« w ^'^'l (37) 

^(att) oc fei(xj). (38) 
This is the transformation used in the paper. 

D.2 Proof of Theorem 5 

Proof. Let {fi>i%j(%j)} be the set of messages at the fixed point and let II be the transformation 
of messages defined by the fixed point messages. We parameterize the messages by T)*^ = 
)• It is enough to prove the assertion after the transformation and in this pa- 
rameterization, because these operations cause similar linearization matrices. 

After the transformation, the LBP update is given in terms of fj as follows: 

_ t+1 _ Ex, ^]t{+,Xi)^i{xi) ILeiVAj Vk^ijxi) 

^ Ex, ^Jt(-i Xi)Mxi) UkENi\j Mfc-S-i(Zi) 
M+. + ) TT - .( ) 1 M+-) 

IlfceAf s \i Vk^i(xi) + '^.(l) ) 
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Figure 4: The sum of indexes is one. 



Figure 5: The sum of indexes is still one. 



Let fj°° := 11(77°°), then fj™ = 1 for all eefi.We can compute f'(fj°°) as follows: 

dfi t+1 

T'(fj°°^ - 3 



dfj\ 



Tp=l 



r bji(+,+) 6,i(-,+) 



V bj(+) 
Xi 3 - m,m 3 
1 — m? 



□ 



E Idea and proof of Theorem 
E.l Idea of Theorem^] 

In Theorem|7l we show that the sum of indexes is equal to one. This is not so special. The simplest 
example that illustrate the idea of the theorem is sketched in figure [4] For each stationary point, 
plus or minus sign is assigned depending on the sign of the second derivative. When we deform the 
function, the sum is still equal to one as long as the outward gradients are positive at the boundaries. 
(See figure|5]) 

Lemma 1, combined with Lemma 2, describes the behavior of the Bethe free energy near the bound- 
ary of L(G). 



E.2 Proof of Lemma 1 



Proof. First, note that it is enough to prove the assertion when hi — and Jy = 0. 

We prove by contradiction. Assume that ||VF(g„)|| oo. Then, there exists R > such that 
|| Vi^QVi) || < R for infinitely many n. Let Bq(R) be the closed ball of radius R centered at the 
origin. Taking subsequences, if necessary, we can assume that 

VF(q n )^^(t) eB (R), (39) 



<nj 

because of the compactness of Bq(R). Let 6^ (sr,, Xj) and b\ n \xi) be the pseudomarginals corre- 
sponding to q n . Since q n — > G dL(G), there exist ij 6 E, Xi and Xj such that 

b^(xi, Xj )^0. 

Without loss of generality, we assume that Xi = +1 and Xj = +1. From d39l , we have 

1 b {n) (+ +)b {n) ( ) 

Therefore b^\+, — ) — > or b\™\—,+) — > holds; we assume b\™\+, — ) — ► without loss of 
generality. Now we have 

&(»)(+) = &(?)(+,_) + 6 W(+,+)_,0. 
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In this situation, the following claim holds. 
Claim. Let k G Ni. In the limit qf n —¥ oo, 

El kjfe { x ii x k) , 
Xi,x k =±l °i K^i) 

converges to a finite value. 

proof of claim. From b\ n \+) — > 0, we have 



4 ) (+,+)4 ) (+,-)4 ) (-) 2 
4 ) (-,+)4 ) (-,-)4 ) (+) 2 



(41) 



4" ) (+,-),4" ) (+ I +)^0 and &<">(-) 



Case 1: &&>(-, +) — ► &* fe (- +) ^ and &&>(- 
In the same way as (l40l i. 



, 4 ) (+,+)4 ) (-.-) 

VF(q n )ik = - log -^V —t^-t 

4 4 ) (+,-)4 n) (- ) +) 



Therefore 



4 } (+,+) , 
4 n) (+,-) 

Then we see that (T4Tb converges to a finite value. 

Case 2: &&>(_ +) _^ 1 and _) _^ . 

Similar to the case 1, we have 

4 ) (+,+)4 ) (-,-) 



7^0. 



4 n) (+>-) 



Therefore 



0. This implies that 



&& } (+.+) 



^0. 



1. Then we see that (|4"TT i converges to 



0and^ l) (-,-)^l. 



4 ) ( a: ^ :z: fe) 
4^) 



□ 



(42) 



finite value 

Case 3: &&>(-,+) 
Same as the case 2. 

Now let us get back to the proof of Lemma 1 . We rewrite (fT~9b as 

VF(q n ) t = \ log^ l) (+) - \ bg &<">(-) + \Y. E * l0 § 

From d39l l, this value converges to £j. The second and the third terms in ( f42b converges to a finite 
value, while the first value converges to infinite. This is a contradiction. □ 

E.3 Detailed proof of TheoremE] 

Proof. Define a map $ : L(G) ->■ R N+M by 

= (1 -di)]- ^2 Xilogb^Xt) + j ^2 ^2 x l logb ik (x il x k ), (43) 



x i = ±l 



keNi Xi,x k =±l 



(44) 



,Xj=±l 



where bij(xi,Xj) and bi(xi) are given by q = {m,i,Xij} S L(G). Therefore, we have VF 



and V$ = V -F. Then following claim holds. 
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Figure 6: The graph G. 

Figure 7: The graph G. 



Claim. The sets $ -1 (0) C L(G) are finite and 

J2 sgn(detV$( ( 7)) = ^ sgn(dct V$(g)), (45) 

?6«- 1 (5) 36*- l (0) 

Before the proof of this claim, we prove Theorem|7]under the claim. 

From (l43l > and d44l i. it is easy to see that = g = {m, = 0, = 0}. At this point, 
we can easily check that V$ — W 2 F is a positive definite matrix. Therefore the right hand side 
of d45l ) is equal to one. The left hand side of d45l > is equal to the left hand side of (??), because 
q E <=> VF(^) = 0. Then the assertion of Theorem|7]is proved. □ 

Proof of the claim. First, we prove that $ _1 ((j)) = (VF) _1 (0) is a finite set. If not, we can choose 
a sequence {q n } of distinct points from this set. Let L(G) be the closure of L(G). Since L(G) is 
compact, we can choose a subsequence that converges to some point G L(G). From Lemma 1, 
<7* G L(G) and V-F (g*) = hold. By the assumption in Theorem|7] we have det V 2 F(g*) ^ 0. 
This implies that VF(q) / in some neighborhood of g» . This is a contradiction because q n — > . 

Secondly, we prove the equality ( |45T > using Lemma 2. Define a sequence of compact convex sets 
C n ■= {q e L(G)\ J2ij£E Y^xi,x ~ ^ n}, which increasingly converges to L(G). Since 

$ _1 (0) and ( 1 >_1 (j) are finite, they are included in C n for sufficiently large n. Take K > and 
e > to satisfy if - e > || (j) ||. From Lemma 1, we see that $(<9C„) n B {K) = 4> for sufficiently 
large n. Let n Q be such a large number. Let II e : R Ar+M — > Bq(K) be a smooth map that is identity 
on Bq(K — e), monotonically increasing on ||x||, and n c (a;) = for ||a;|| > K. Then we obtain 

a composition map $ := II e o $ : C„ -> B (K) that satisfy $(<9C„ ) C dB (K). By definition, 
we have fc-^Q) = $ _1 (0) and = Therefore, both and (j) are regular values 

of <t. From Lemma 2, we have 

sgn(det V$(g)) = sgn(det V$(g)). 

Then, the assertion of the claim is proved. □ 

F Proof of Corollary! 
F. 1 Detailed proof of Example 2 

In this subsection we prove the assertion of Corollary [4] for the graph of Example 2, which is dis- 
played in figure[6] The + and — signs represent that of two body interactions. 
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Figure 8: Two other types of graphs. 



It is enough to check that dct(7 - BM) > for arbitrary < #13, /3 2 3, fiu, ftu < 1 and -1 < 
/?i2 < 0. The graph G in figure [7] is obtained by erasing vertices 2 and 4 in G. To compute 
det(I — BM), it is enough to consider G. In fact 



det(J - BM) = (ciPr 1 

= Hi 1 -9(f)) 



(46) 
(47) 



peP 

Cg^)- 1 = det(I-RM), 



where /3 ei := /3x 2 ^23, Pe 2 ■= #L3> /3e 3 : = /3i4/?34 and /3 ei = /3 e - ; . The equality between (06]) and 
(l47l i is obtained by the one to one correspondence between prime cycles of G and G. 

By definition, we have 



BM = 



where the rows and columns are indexed by ex, e%, 63, e%, e 2 and 63. Then the determinant is 
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det(J - = dct 



/3 ei /? ei 



I- 



Pe 2 





dct 




/?e 3 Pe 3 

(1 Pe\Pe 2 ~ PeiPe 3 ~ Pe 2 Pe 3 2/3 ei /3 e2 /3 e3 ) 



Since — 1 < /3 ei < and < /3 e2 , /3 e3 < 1, we conclude that this is positive. 



F.2 Other cases 

There are two operations on graphs that do not change the set of prime cycles. The first one is adding 
or removing a vertex of degree two on any edge. The second one is adding or removing an edge 
with a vertex of degree one. With these two operations, all graphs that have two linearly independent 
cycles are reduced to three types of graphs. The first type is in figure|7] The other types are in figure 

m 

Up to equivalence of interactions, all types of signs of two body interactions are listed in figure [9] 
except for the attractive case. We check the uniqueness for each case in order. 

Case (1): Proved in Example 2. 
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Figure 9: List of interaction types. 
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Case (2): In this case, 
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BM 



where rows and columns are labeled by e±, e2, e3, ei, e2 and e3. Then the determinant is 

det(7 - BM) = (1 - /3 ei )(l - /3 e3 )(l - /3 6l - /3 e3 + Peje 3 - 4/3 £l /3 e 2 2 & 3 ) • (48) 

This is positive when < j3 ei , f3 e2 < 1 an d — 1 < Pe 3 < 0. 

Case (3): The determinant d48b is also positive when < f3 e2 < 1 and — 1 < (3 ei , (3 e:i < 0. 
Case (4): In this case, 



BM 



where rows and columns are labeled by e±, e2, e\ and e<i. Then we have 

det(J - BM) = (1 - /3 ei )(l - p e2 )(l - ei - f3 e2 - 3/3 ei /3 e2 ). (49) 
This is positive when < f3 ei < 1 and — 1 < f3 e2 < 0. 
Case (5): The determinant d49l is positive when —1 < (3 ei , (3 £2 < 0. 
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